Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update MSG-4 visible calibration coefficients #81

Merged
merged 9 commits into from
Nov 16, 2023
Merged

Update MSG-4 visible calibration coefficients #81

merged 9 commits into from
Nov 16, 2023

Conversation

sfinkens
Copy link
Collaborator

@sfinkens sfinkens commented Aug 7, 2023 •
edited
Loading

Update MSG-4 visible calibration coefficients to latest version from May 2023: https://msgcpp.knmi.nl/solar-channel-calibration.html

Edit 1

For some reason creating the conda environment on macos failed (see https://github.com/foua-pps/level1c4pps/actions/runs/5785938956):

ImportError: dlopen(...libmambapy/bindings.cpython-311-darwin.so, 0x0002): Library not loaded:
...
Error: The process '/Users/runner/miniconda3/condabin/mamba' failed with exit code 1

Switching to mambaforge seems to solve the problem.

Edit 2

Now creating the conda env with Python-3.8 takes forever...

I noticed some warnings about deprecated node 12 (https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/) so I updated actions/checkout and codecov/codecov-action to version 3. But that doesn't speed things up...

  • Tests added
  • Tests passed: Passes pytest level1c4pps
  • Passes flake8

@codecov-commenter
Copy link

codecov-commenter commented Aug 7, 2023 •
edited by codecov bot
Loading

Codecov Report

Merging #81 (1c8cc0f) into main (1ed7063) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #81   +/-   ##
=======================================
  Coverage   77.61%   77.61%           
=======================================
  Files          21       21           
  Lines        1452     1452           
  Branches      107      120   +13     
=======================================
  Hits         1127     1127           
  Misses        301      301           
  Partials       24       24
Flag Coverage Δ
unittests 77.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
level1c4pps/calibration_coefs.py 100.00% <ø> (ø)
level1c4pps/tests/test_seviri2pps.py 96.76% <ø> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@ninahakansson
Copy link
Collaborator

Can we just remove support for python 3.8? Or do you still need to run level1c4pps for python-3.8?

@sfinkens
Copy link
Collaborator Author

Can we just remove support for python 3.8? Or do you still need to run level1c4pps for python-3.8?

Forwarding this to @yeigenbrodt 🙂

@yeigenbrodt
Copy link
Contributor

Can we just remove support for python 3.8? Or do you still need to run level1c4pps for python-3.8?

I see no reason to support 3.8.

@sfinkens
Copy link
Collaborator Author

sfinkens commented Aug 28, 2023 •
edited
Loading

I simplified the satpy version requirement a little (> 0.41), which speeds up the build a bit:

OS Python Build time old [min] Build time new [min]
ubuntu 3.9 inf 22
ubuntu 3.10 116 11
ubuntu 3.11 13 3
macos 3.9 inf 46
macos 3.10 37 22
macos 3.11 14 8

So at least from the Python-3.11 jobs you get a quick feedback. But 46 minutes for Python-3.9 is way too long... Any idea how to speed this up?

@mraspaud
Copy link
Collaborator

I see this is already using mamba, which I was going to suggest. I'm very surprised it takes such a long time with mamba though...
As for ways to improve this, maybe the requirements file should be trimmed down? Satpy has hard requirements on pyresample, dask and xarray at least, so these can go.

@sfinkens
Copy link
Collaborator Author

sfinkens commented Sep 11, 2023 •
edited
Loading

As for ways to improve this, maybe the requirements file should be trimmed down?

That helped a lot! Now at 6 minutes max 🙂

@ninahakansson From my point of view this is ready to merge now.

@ninahakansson ninahakansson merged commit 2568eef into foua-pps:main Nov 16, 2023
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ninahakansson ninahakansson Awaiting requested review from ninahakansson

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sfinkens @codecov-commenter @ninahakansson @yeigenbrodt @mraspaud